A New Prediction Oriented Barrier Synchronization on SMP Clusters

Clusters of Symmetric Multiprocessors (CSMP) are becoming an increasingly popular high-performance computing platform due to the commodity availability of multiprocessor nodes, mature SMP operating systems, low-latency, highbandwidth data networks, and superior price-performance ratio. Fast synchronization is crucial to making efficient use of SMP clusters. In this paper, we focus on one kind of synchronization in parallel computing, SMP cluster barrier operations (SCB). Typically, parallel programs either for shared-memory (SMP) or for CSMPs alternate between computation and communication supersteps. Thus, the processors synchronize between each step, arriving in close temporal proximity at the barriers. This observation encouraged us to design a new barrier algorithm with faster execution than traditional approaches while preserving scalability and portability. Our new CSMP barrier algorithm PSCB consists of two levels, one for each SMP called the node barrier (NB), and the other for the overall cluster called SMP cluster barrier (MPB). We improve the SCB by using predictive techniques on each SMP node to overlap the NB and MPB barrier levels. This method provides an efficient technique to overlap different communications involved on either side of the barrier. Our work includes an experimental study of these barrier operations on several parallel architectures. These results support our claim of a faster barrier synchronization. KeywordsSMP, hybrid SMP clusters, barrier, prediction.